Group experience user interface

ABSTRACT

The description relates to a shared digital workspace. One example includes a display device and sensors. The sensors are configured to detect users proximate the display device and to detect that an individual user is performing an individual user command relative to the display device. The system also includes a graphical user interface configured to be presented on the display device that allows multiple detected users to simultaneously interact with the graphical user interface via user commands.

BACKGROUND

Traditionally, computing devices have been configured to be operated bya single user at a time. The computing devices generated a graphicaluser interface directed at the individual user. If a second user wantedto use the computing device, the first user logged out and the seconduser logged in. Responsively, the computing device would generate asecond graphical user interface directed to the second user. Thus, thecomputing device is configured for sequential use rather thansimultaneous use by the users.

SUMMARY

The description relates to a shared digital workspace. One exampleincludes a display device and sensors. The sensors are configured todetect users proximate the display device and to detect that anindividual user is performing an individual user command (e.g., inputcommand) relative to the display device. The system also includes agraphical user interface configured to be presented on the displaydevice that allows multiple detected users to simultaneously interactwith the graphical user interface via user commands.

The above listed example is intended to provide a quick reference to aidthe reader and is not intended to define the scope of the conceptsdescribed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate implementations of the conceptsconveyed in the present document. Features of the illustratedimplementations can be more readily understood by reference to thefollowing description taken in conjunction with the accompanyingdrawings. Like reference numbers in the various drawings are usedwherever feasible to indicate like elements. Further, the left-mostnumeral of each reference number conveys the FIG. and associateddiscussion where the reference number is first introduced.

FIGS. 1-16 show example shared digital workspace scenarios in accordancewith some implementations of the present concepts.

FIGS. 17-18 show example shared digital workspace systems in accordancewith some implementations of the present concepts.

FIG. 19 is a flowchart of example shared digital workspace techniques inaccordance with some implementations of the present concepts.

OVERVIEW

The description relates to a shared digital workspace or “sharedcanvas”, which may be provided by a digital whiteboard, among otherdevices. As such, these devices can be thought of as group experiencedevices. The shared canvas can allow multiple simultaneous users (e.g.,a group of users). Any device can potentially function as a groupexperience device, but the concepts are more readily applied to devicesthat have large enough displays to be readily physically engaged byseveral users simultaneously.

“Group” for the purpose of this discussion is a collection ofindividuals (at least some of whom) are physically co-located in thesame space and are interacting with each other. One group example canconsist of a couple of employees meeting together to discuss or debate atopic. A family interacting in their kitchen can be another such group.A few friends getting together to plan an activity can be yet anothergroup.

The present concepts can provide a group experience. The groupexperience can be centered around a single shared device (e.g., groupexperience device) that all of the group members can comfortablyinteract and control. In the group experience, no single user isdominating the group experience device. The ability of each member ofthe group to initiate an interaction can be immediate and frictionless.The group experience device is often the focal point for the discussionamong the group members and while one user interacts, the others canwatch and can immediately step in and intervene if they feel like. Thus,two or more users can interact with the shared canvas simultaneously.

In some of the proposed implementations, group experience devices canbecome a category of their own—as a peer to desktops and tablets.Differentiated from their peers, group experience devices can beoptimized for the group experience instead of the single user. The groupexperience devices can employ different, yet familiar, form factors fromsingle user devices. The form factors can include innovative interactionmodels and input technologies. Input technologies such as, touch, pen,gesture, speech and/or vision can be leveraged in the group experiencedevices. In some cases, a device may be configured to operate in eitheran individual-centric mode or a group mode, as desired.

The applicability of the group experience devices is broad—virtually anyspace where people likely meet and interact. Meeting rooms, offices,home kitchens, family rooms, and even hallways are all great locationsto place them.

The group experience devices can leverage the natural way humansinteract with each other in a group settings. For instance, the groupexperience devices can support and enable our natural stance, thetypical dynamics between members of the group, and the type ofactivities that a group is likely to engage in together.

Some of the present implementations can emphasize anonymous usage overauthentication. These implementations can strongly push a highly commonexperience and common organization at the expense of personalization andpersonality. Such a configuration can be attractive to users for theirlow barrier to use; just walk up and start using the group experiencedevice. However, these same users may yearn for the ability to integratetheir personal data into a group experience. Some of the presentimplementations can address both of these considerations.

Toward this end, some implementations can offer two layers offunctionality. The first layer can be thought of as a basicfunctionality. Any user can engage the group experience device andimmediately utilize the basic functionality. The second layer can be aconditional or advanced functionality. For instance, the secondfunctionality can be available only to users who have been identified(e.g., authenticated) by the group experience device. These aspects areexplained below by way of example.

SCENARIO EXAMPLES

FIGS. 1-15 collectively show a group experience scenario 100 where agroup experience device 102 is positioned in a conference room. In thisexample, two users 104(1) and 104(2) walk into the conference room. Asthe users are detected, the group experience device 102 can power up toa ready state and present a shared display area or shared canvas 106. Inthis example, the first user 104(1) immediately walks up and startswriting on the shared canvas 106. In this example the user writes theword “jaguar”. Writing on the shared canvas 106 can be enabled as anexample of a basic functionality that is made available to all users.

Other examples of basic functionality can include images, video, textboxes, shape recorded smart objects from ink, lists, tables, timelines,shape art (circle, square, triangle, etc.), sticky notes, and/or charts,among others. Still further examples of basic functionality can includea brainstorming application, a research and exploration tool,time/calendar coordination, activity planner, product/shoppingcomparison tool, data exploration tool, budgeting and allocations, amongothers. Further, in some implementations the basic functionality of theshared canvas can be limited to the visible canvas. In otherconfigurations, the basic functionality can include a scrollable canvasin the horizontal and/or vertical directions (e.g., the available canvascan be greater than what can be displayed at a given instance).

Further, in this case, assume that each of the users is identified.Techniques and mechanisms for user identification are described belowrelative to FIGS. 17-18. Assume that the first user 104(1) has a pictureof a jaguar in her personal data that she wants to show on the sharedcanvas 106. The personal data may be stored in the cloud, such as in aglobal account and/or on one of her personal devices, such as a smartphone, tablet, or other computer. Access to personal data is an exampleof the conditional functionality.

As an identified user, the first user 104(1) can access her personaldata through the shared canvas 106. Some implementations can allow theuser to have a personal space on the shared canvas. The user caninteract with their personal content in their personal space. Briefly,in this implementation the personal space is manifest as a portal ortunnel, which is described below.

In this particular implementation, the first user can make a specificuser command gesture, such as a double-tap on the shared canvas toaccess her personal data. Specifically, the group experience device 102can identify the user and determine that that specific user hasperformed a user command to access his/her personal data. The groupexperience device can utilize the identity of the user to access theuser's personal data. Note that the double-tap is only one example forcontrolling the shared canvas. For instance, other implementations canuse other active user commands and/or passive user commands, such astouch gestures, non-touch gestures, eye gaze, voice commands, and/orcommands entered via another device, such as smart phone, among others.Still other implementations can automatically open a portal for theidentified user rather than awaiting a user command.

As represented in FIG. 2, in this example, the user double-tap gesturecan cause a portal or tunnel 202 to be presented on the shared canvas106. The portal 202 can represent that user's personal space/data on theshared canvas. The portal can be identified as belonging to the user(e.g., user 1). (Of course, other ways of associating the user and theportal can be employed. For instance, the photograph of the user or anavatar of the user may be displayed proximate to the portal.) The usercan perform another user command relative to the portal 202 to view oraccess his/her personal data. For instance, the user may touch theportal.

FIG. 3 shows the user's personal data/content inside the portal 202. Inthis case, the personal data is the user's pictures. (Of course, due tothe constraint of the drawing page only a representative example of theuser's personal data can be illustrated. For instance, the portal mayshow the user's global profile (e.g., cloud stored data) and/or data byindividual device. Further, the user's personal data may be organized invarious ways for the user in the portal. The user may also have theoption of searching the personal data in the portal.) Someimplementations can allow the user to zoom a portion of the sharedcanvas. For instance, the user could zoom on her portal so that ittemporarily occupied a greater portion or all of the shared canvas tofacilitate the user viewing her personal data.

As shown in FIG. 4, the user 104(1) can easily retrieve data from theportal 202 and move a copy to the shared canvas 106. In this case, theuser drags a picture of a jaguar 402 from the portal 202 to the sharedcanvas 106.

The portal 202 can be the individual user's private space on theotherwise shared canvas 106. The portal can be interactive and canaccept any input that the user might perform on one of her devices shownin the portal. This allows for bi-directional input between this portaland the personal device (and/or the user's global content). As mentionedabove, in some configurations, users can temporarily make any portal 202at any time take up the full screen real estate of the shared canvas106. Alternatively or additionally, some implementations can allow theuser to define how she wants her portal displayed in terms of sizeand/or content. For instance, the user may want the portal to employsemantic view techniques where the user can either zoom out of a view tosee a summary of the content or zoom in to see content grouped byvarious parameters, such as category, alphabetic, or chronologicalparameters, among others.

FIG. 5 shows how the second user 104(2) can utilize the basicfunctionality, such as writing the word “lions”. FIG. 6 shows the seconduser launching his own portal 202(2).

FIGS. 7-8 show the second user 104(2) selecting and moving personalcontent from his portal 202(2) to the shared canvas 106. In this case,the content is an article about lions indicated at 802.

Note also, that in FIG. 8, the first user's portal 202 has closed toprotect the user's privacy. The first user 104(1) can manually perform agesture (or other user command) to cause the portal 202 to close or theportal can close automatically. For example, the portal can close basedupon a period of inactivity or in an instance when the user walks awayfrom the portal, such that it is apparent the user is not intending toengage the portal 202. The portal is still readily available to the uservia a simple user command. In another implementation, the first user canbe automatically notified when other users (such as the second user)approach the group experience device 102 when the first user has herportal 202 open. Such a configuration can allow the user to take actionsto safeguard her personal content if she so desires.

At any point either user 104(1) or 104(2) can make a copy of the sharedcanvas 106 and store the copy in their personal data (e.g., via theirportal).

FIGS. 1-8 introduce shared canvas features relative to the users workingindependently, either serially or simultaneously. These features canalso facilitate collaboration between the users as will be explainedbelow by way of example.

FIG. 9 shows a subsequent scenario where the users 104(1) and 104(2) areusing their previously described content in a collaborative effort(e.g., collaborative manner). The second user 104(2) adds a title to thecollaborative effort of “Our cat-family research project” at 902 whilethe first user 104(1) simultaneously adds text that “Many members of thecat family are threatened by habitat loss.” at 904. Thus, in this caseeach user has used their portal to bring their own personal content intothe shared space and combined the personal content with content createdon the shared space to accomplish the collaborative effort. In thisexample, some of the work was performed in a concurrent manner whileother work was performed in a simultaneous manner (e.g., multiple usersworking on the shared canvas at the same time).

Further, some implementations can allow a remote user to participate inthe shared session (e.g., on the shared canvas). For instance, a noticecould be presented on the shared canvas 106 that a “third user” wants toparticipate remotely. Either of the first or second users 104 couldindicate that they want to allow the remote user to participate. Theremote user could then see a representation of the shared canvas. FIG.10 shows an example of a representation 1002 of the shared canvas on theremote user's device 1004 from the perspective of the remote user 1006.

FIG. 11 shows a subsequent representation on the remote user's device1004 where the remote user has added content to the shared canvas viathe remote device. In this case, the remote user added the words “andpoaching” so that the sentence now reads “Many members of the cat familyare threatened by habitat loss and poaching.” Various techniques can beemployed to indicate which user contributed which content. In this case,the remote user's content is shown in italics whereas the first user'scontent is shown in standard text. Of course, other techniques can beutilized to distinguish users' contributions. For instance, differentcolors of text may be associated with different users.

FIG. 12 shows shared canvas 106 with the addition of the content addedby the remote user. Further, in this case, the remote user's photo 1202appears on the shared canvas proximate to his content so that the otherusers can readily determine what the remote user added. In this casethere are two indicators: first, the remote user's text is different(e.g. italicized) and the remote user's image is shown by the content.Of course, other techniques for identifying a specific user's contentcan be employed.

FIG. 13 shows a subsequent instance where the first user left theconference room and the remote user disconnected or disengaged from theshared session. Responsively, the first user's portal is removed fromthe shared canvas 106. Similarly, the remote user's photo is removed.However, there can be persistent identification of which usercontributed which content to the shared canvas.

FIG. 14 shows the shared canvas 106 as the second user leaves the roomand his portal is removed from the shared canvas. In someimplementations, a copy of the shared canvas can be automatically storedfor the users so that the next time they enter the room the sharedcanvas is returned in its previous state. In summary, FIGS. 1-14 showhow the shared canvas can allow multiple users to collaborate on aproject (or other content). The collaboration can be serial and/orsimultaneous.

FIG. 15 shows a subsequent view where the content of the shared canvas106 has been deleted and the group experience device 102 can await moreusers. In some implementations, the group experience device canpositively contribute to the aesthetics of its environment when not inuse. In this case, art is displayed on the group experience device whennot used. In other cases, advertisements can be presented on the groupexperience device when not in use. Still other implementations can powerdown the display when not in use to save power though sensors can remainactive to detect the presence of new users. Of course, the context ofuse can affect what actions are taken when the users are not activelyusing the shared canvas. An alternative example is shown and describedbelow relative to FIG. 16.

FIG. 16 shows a shared canvas 106(1) on a group experience device 102(1)in a home environment. In this case, the members of the family can bethe users. In this context the family may want the shared canvas 106(1)to persist indefinitely until they decide to delete it or to temporarilyor permanently display something else. For example, the shared canvas106(1) can be persistent, but the family may intermittently use thegroup experience device 102(1) to watch video content (such as TV) andthen switch back to the shared canvas 106(1). In a family-centric usecase, sometimes the family members may engage the shared canvas at thesame time. At other times, one family member may add or delete somethingto the shared canvas or simply view the shared canvas. Regardless, theshared canvas can be visible to, and/or engaged by, the family membersas they pass by.

To summarize, some implementations can include a large screen thatblossoms from its ambient, low-power state into a clean shared canvas,ready to turn on, as soon as a person's local existence is detected bythe group experience device. This shared canvas can be thought of as theGUI shell of a group experience operating system. The GUI shell canfunction as a content container as well as an application container. Itcan act as both a surface for new content creation as well as a surfacefor “tunnels” or “portals” to expose other devices' desktops andapplications. It can also be where users interact with applicationsrunning locally on the large screen group experience device.

Examples of portals or tunnels can include interactive projection of anapplication and/or a desktop of a personal companion device in the room.Another example can include an interactive projection of an applicationand/or a desktop of a remote, personal companion device. Still otherexamples can include an application running on a local device and/orapplications running in the cloud.

Any detected user can use the shared canvas. Attempts can be made toauthenticate the detected user (e.g., identify the user with a degree ofconfidence). In some implementations, authenticated users are indicatedby name and picture on the shared canvas. Their associated devices canbe grouped under their picture if they open a portal. Theseauthenticated devices can be “tunneled” onto a sub-window on the sharedcanvas at any time. The sub-window (e.g., portal or tunnel) can be theuser's personal space on the shared canvas. While specific portalexamples are illustrated and discussed above, the inventive concepts arenot intended to be limited to these examples. Instead, a portal cansimply be a way for a user to access private content on the sharedcanvas paradigm.

FIGS. 17-18 relate to systems for accomplishing group experienceconcepts. These systems can include a display and sensors for detectingand identifying users and for identifying user commands (e.g., controlgestures). Specific system examples are introduced followed by adiscussion of other system display and sensor configurations.

FIG. 17 shows an example group experience system 1700. In this case, thesystem includes a group experience device 102(2) manifest as a touchscreen or touch display and two sets of sensors 1702 and 1704. Theshared canvas 106(2) can be presented on the touch display. In thiscase, the first set of sensors 1702 can be oriented to sense an area infront of the shared canvas 106(2). This orientation is intended todetect users as they enter the room. The second set of sensors 1704 canbe oriented to sense an area directly in front of the shared canvas106(2). This orientation is intended to detect user's engaging theshared canvas 106(2) with user commands. In this case, the display andthe two sets of sensors are connected to a computing device 1706 (inthis case not visible to the users) which performs processing for agroup experience graphical user interface of the shared canvas which ispresented on the group experience device 102(2).

The first set of sensors 1702 can detect that a user or users haveentered the room in which the group experience device 102(2) ispositioned (or have otherwise entered an area of interest). In thiscase, the first set of sensors includes two outwardly facing subsets1702(1) and 1702(2) positioned above and below the group experiencedevice 102(2) to detect user gestures, touch and/or eye gaze direction.Note that the number, position, and/or orientation of the first andsecond sets of sensors 1702 and 1704 is provided for purposes of exampleand is not intended to be limiting. It is contemplated that many othersensor arrangements can be configured to accomplish the presentconcepts. Further, while distinct and conspicuous sensors areillustrated, it is contemplated that the sensors will become smaller andless evident to the user as the technology matures.

The first and second sets of sensors 1702 and 1704 can be cameras, suchas arrays of cameras. The cameras can be configured for visible light,infrared, and/or other frequencies. The cameras may operate incooperation with an infrared pattern projector that can aid the camerasto distinguish objects from one another. Other camera configurations mayemploy time of flight or other techniques to enhance informationcaptured by the cameras about the users and/or the environment aroundthe group experience device 102(2). In one implementation, the first andsecond sets of sensors can be Kinect™ brand sensing technology offeredby Microsoft® Corp.

Once users have been detected, the first and/or second sets of sensors1702 and 1704 can employ biometrics, such as face tracking and facerecognition/identification to attempt to identify the users. Forinstance, tracked biometric features can be compared to a database ofknown users for a potential match. Simultaneously, the users' positionrelative to the shared canvas 106(2) and movements can be tracked todetermine if the user is attempting a user command, such as writing onthe display, making a control gesture, etc. At this point, the basicfunctionality is available to the user. If the user(s) are identifiedthen secondary functionality can be made available to individual users.

In some implementations, user identification can be associated with aconfidence score or level (e.g., the detected user is ‘user 1’ with aconfidence of 70%). As such, individual features of the secondaryfunctionality can specify a specific confidence threshold. For instance,a specific application program interface (API) may have a confidencethreshold of 40% while another API may have a confidence threshold of80%. For example, one API may be configured so that the threshold is 50%for a user on the shared canvas 106(2) to access personal photos ordocuments through a portal, while another API may require a 90%threshold for the user to access financial information (e.g., transferfunds or access credit card numbers) through the portal.

Further still, rather than a one-time process, in the group experiencesystem 1700, user identification may be an ongoing process where thesystem continuously monitors for user commands and attempts to identifywhich user is performing the user command. Thus, the ongoing acts ofusing the group experience device 102(2) can be what it authenticated(e.g., a user command was detected and in analyzing the detection, it isdetermined what user performed the user command). Thus, from oneperspective, some implementations can monitor for user commands. Theimplementations can attempt to identify the user through the usercommand, such as from biometrics detected in the act of detecting thecommand. Thus, these implementations can continually detect usercommands and attempt to continually identify users via the commands.

Note also, that each of the users in FIG. 17 has a personal device 1708and 1710, respectively (in this case a smart phone). The personal devicecan be utilized in user detection and identification. The personaldevice 1708 and 1710 can work alternatively or additionally to the firstand second sets of sensors 1702 and 1704 to help identify the usersand/or detect user commands. For example, the personal device could sendout an identification signal, like a beacon signal that can be used bythe system to identify the user.

The personal device 1708 and 1710 may be leveraged in other ways. Forinstance, the personal device may send a capacitive signal through theuser's body that can be detected when the user touches or is proximateto the group experience device 102(2). This feature can be leveraged asmultipath capacitive touch. For instance different fingers havedifferent length paths. Some implementations could also add a conductivestrip on the floor below the group experience device 102(2), so thatuser engagement of the group experience device 102(2) completes acircuit through the group experience device and the conductive strip.Capacitance pens can be utilized in a similar manner, especially ininstances where users have their own personal pens that are uniquelyidentifiable. Thus, pen 1712 can serve dual purposes as a digitalwriting instrument and to identify user 104(1).

In a similar manner, when a touch user command is detected as designatedat 1714, fingerprint (and/or other biometric) analysis can be used toidentify the user performing the command. Fingerprint analysis can alsoidentify which individual finger of the user is touching the displayscreen and the orientation of the finger. This information can beutilized in various ways. For instance, if two users simultaneouslytouch a region of the board, the finger orientation (determined throughthe fingerprint analysis) can indicate which finger belongs to whichuser. For instance, fingerprints tend to be elongate along a length ofthe user's finger. Thus, the fingerprint can indicate the orientation ofthe user's finger and arm.

FIG. 18 shows an alternative group experience system configuration. Thisgroup experience system 1800 includes a group experience device 102(3)coupled to another computing device 1706(1). The other computing devicecan be embodied as other computers associated with individual usersand/or computers that provide other functionality (e.g., search engines,user profile storage, etc.). The group experience device 102(3) caninclude a display 1802, a processor 1804, storage/memory 1806, sensors1808, a communication component (hardware and/or software) 1810, adetection/identification component 1812, and a group experienceapplication/operating system 1814. The group experience device 102(3)can alternatively or additionally include other elements, such as,buses, graphics cards (e.g., graphics processing units (CPUs), networkhardware), etc., which are not illustrated or discussed here for sake ofbrevity.

Note further, that the present implementations are not limited to aspecific type of display 1802. In contrast, workable implementations canbe accomplished with projection displays, light emitting diode (LED)displays, liquid crystal displays, electroluminescent displays, plasmadisplays, and/or other developing or yet to be developed display types.

In this case, the display 1802 includes two sets of sensors 1808 in theform of cameras. A first set of cameras 1808(1) point away from thedisplay 1802 and a second set of cameras 1808(2) point parallel to thedisplay surface to sense user control gestures. The second set ofcameras can allow the display to function as a touch display withoutactually having a touch sensitive surface (e.g., that senses a user'sphysical touch). In some implementations, the first set of cameras1808(1) can include two sub-groups. The first subgroup can be configuredto capture a “near field” space directly in front of the groupexperience device 102(3). The second subgroup can be configured tocapture a “far field” space that is more distant from the groupexperience device. The far field subgroup can be used to detect usersentering a room in which the group experience device is located. Thenear field subgroup can be configured to capture biometric data from theusers as they engage the shared canvas. The two sets of cameras can beoperated cooperatively by the detection/identification component 1812 todetermine who the users are, and which user is performing a user commandby touching which portion of the display 1802 (or via other usercommand(s)).

The group experience application/operating system 1814 can generate theshared canvas on the group experience device 102(3). The groupexperience application/operating system 1814 can allow user interactionof the shared canvas based upon information obtained from thedetection/identification component 1812.

In some implementations, the shared canvas can be thought of as beingautonomous from the users. The users can be thought of as guests on theshared canvas and the users have equal standing relative to the sharedcanvas. Thus, in at least some implementations there is no controllingor primary user. In other implementations, the shared canvas can beassociated with an individual user (e.g., primary user), such as a userthat launches the shared canvas. In this latter configuration, the otherusers can be secondary to the primary user relative to that instance ofthe shared canvas.

While the illustrated second set of cameras sense along a surface of thedisplay 1802, other camera configurations can be employed such as thosethat image through the display. One suitable camera for such aconfiguration is a wedge type camera that could be positioned in frontof or behind the display or to the side of the display. This type ofconfiguration can detect the user's fingers touching the display and canalso look at the user's eyes to determine where on the display the useris looking. Biometric information obtained by the cameras can beutilized by the detection/identification component 1812 to identify theuser touching the display. Alternatively or additionally, thisinformation can be interpreted as a user command. For instance, wherethe user is looking at on the shared canvas (e.g., user gaze) can beinterpreted as a user command relative to content at that location, forexample.

While not illustrated, in a corporate setting the system could alsoutilize the user's badge for user identification purposes. For instance,the system could read the name on the badge, correlate the picture onthe badge with the user, and/or scan the badge to obtain information,such as contained on an RFID or other device in the badge.

An alternative configuration can utilize a personal device in the formof eye tracking eyeglasses 1816. The eye tracking eyeglasses can havesensors configured to track the user's eyes and other sensors configuredto detect the environment in a direction that the user is looking. Theeye tracking eyeglasses can send signals to the group experience device102(3), such as via Bluetooth technology, among others, identifying theuser and where on the display 1802 the user is looking.

In summary, user authentication can be achieved through variousmechanisms. The detection/identification component 1812 can analyze theavailable information to detect users and identify those users at aconfidence level. In some cases, the user can be auto discovered andauthenticated through a personal device, such as a smart phone. Forinstance, auto discovery can be accomplished via device discovery. Theuser who is logged into a recognized mobile device and is proximate tothe group experience device 102(3) can appear as a user in the groupexperience system 1800. For instance, the personal device may provide anultrasound signature, network infrastructure, Wi-Fi Direct, opticalsignatures, manual connection, etc. In other cases, the user can beautomatically identified through face recognition, such as using Kinect(e.g., cameras 1808(1)).

In some cases, users may be manually authenticated into the system, suchas via a keyboard, fingerprint scanner, badge, NFC, etc.

A remote user can manually authenticate into the group experience systemby having the URL identifier of the open group experience. An indicationregarding the remote user may be presented on the group experiencedevice 102(3). A user that is present may allow or disallow the remoteuser.

Once the detection/identification module 1812 identifies the user with agiven certainty/confidence additional functionality can be madeavailable. In some cases, APIs can be set up so that only those APIcalls from a user that is authenticated with a certainty above athreshold for that API go through.

The detection/identification component 1812 may continuously orrepeatedly attempt to identify the users. In one configuration, thedetection/identification component can continuously attempt to detectuser commands and identify who is performing the user commands. Thus,every action taken by the user can be part of the authenticationprocess. This can provide frictionless instant authentication based uponeach action/interaction rather than per session. This configuration canalso allow multiple modes at the same time because it can know what useris asking for each one. Viewed from one perspective, in someimplementations, the user can become the glue to all actions. Inessence, the user can frictionlessly engage the group experience device102(3). Individual actions (e.g., user commands) performed by the userare evaluated and identified to the user. Thus, in some implementations,it is the user actions that can be authenticated rather than the user.

From one perspective, group experience device 102(3) can be thought ofas a computer. Processor 1804 can execute data in the form ofcomputer-readable instructions to provide a functionality. Data, such ascomputer-readable instructions and/or user-related data, can be storedon storage 1806, such as storage that can be internal or external to thecomputer. The storage can include any one or more of volatile ornon-volatile memory, hard drives, flash storage devices, and/or opticalstorage devices (e.g., CDs, DVDs etc.), among others. As used herein,the term “computer-readable media” can include signals. In contrast, theterm “computer-readable storage media” excludes signals.Computer-readable storage media includes “computer-readable storagedevices.” Examples of computer-readable storage devices include volatilestorage media, such as RAM, and non-volatile storage media, such as harddrives, optical discs, and flash memory, among others.

In some configurations, group experience device 102(3) can include asystem on a chip (SOC) type design. In such a case, functionalityprovided by the computer can be integrated on a single SOC or multiplecoupled SOCs. One or more processors can be configured to coordinatewith shared resources, such as memory, storage, etc., and/or one or morededicated resources, such as hardware blocks configured to performcertain specific functionality. Thus, the term “processor” as usedherein can also refer to central processing units (CPU), graphicalprocessing units (CPUs), controllers, microcontrollers, processor cores,or other types of processing devices suitable for implementation both inconventional computing architectures as well as SOC designs.

Examples of other computers can include traditional computing devices,such as personal computers, desktop computers, notebook computers, cellphones, smart phones, personal digital assistants, pad type computers,cameras, large display devices, projection devices, or any of a myriadof ever-evolving or yet to be developed types of computing devices.

The user's privacy can be protected by only enabling userdiscovery/identification features upon the user giving their expressconsent. All privacy and security procedures can be implemented tosafeguard the user. For instance, the user may provide an authorization(and/or define the conditions of the authorization) on his/hersmartphone. The smartphone can be detected by other computing devices(e.g., the group experience device). Those devices only proceed with theidentifying the user according to the conditions of the authorization.Otherwise, user information is not gathered. Instead the user can beoffered the basic functionality as a ‘generic user’. Advancedfunctionality may not be enabled for the user in such a scenario.

METHOD EXAMPLE

FIG. 19 shows an example group experience method 1900.

In this case, the method can detect multiple users at block 1902.

The method can present a graphical user interface (GUI) at block 1904.

The method can allow each of the users to simultaneously interact withthe GUI utilizing basic functionality at block 1906.

From one perspective this multiuser GUI can be thought of as a ‘sharedcanvas’ where users create new content. In some implementations, touch,via finger touch (e.g., touch) and/or pen touch (e.g., pen) can be theprimary input mechanisms for creation of new content on the sharedcanvas via basic functionality. Much like a physical whiteboard, a usercan write free-form ink on the canvas. In some cases, the canvas can bescrollable, to create an (essentially) infinite canvas. This ink can beautomatically recognized through a number of shape recognizers, bridgingthe gap between free form ink and objects with schema and behaviors. Forexample, as a user writes a list, the methods can recognize this andstart to draw guide lines for the next item in the list as well asprovide behaviors for ink-to-text, sorting, etc.

The method can attempt to identify individual users at block 1908. Useridentification can utilize biometrics and/or personal devices, amongothers, to identify the user. Specific examples are described in moredetail above relative to FIGS. 17-18.

The method can enable advanced functionality to individual identifiedusers at block 1910. For example, the advanced functionality can includeaccess to personal data of the individual identified users on thegraphical user interface.

The advanced functionality can relate to personal data, such as may beaccessed via tunnels or portals. Content objects can be instantiatedfrom tunnels. For example, the user can take a snapshot of the currentprojection of a mobile device and it can appear as a native image objecton the shared canvas. That image object can contain the metadata (user,machine, date/time, file, application, position) to fully understand theprovenance of the image.

Going in the other direction, content objects can be dragged from theshared space into tunnels (e.g., personal space). This can be enabled asa simple cut and paste operation into the active application of thetunnel (a copy operation), or as embedding of a hyperlink to the contentobject in that canvas (a reference operation).

When a user instantiates a new tunnel that includes either a local orcloud application, the method can know who instantiated it, so it canappear where that tunnel has a clear visual notification that it isrunning under the identity of the user who created the tunnel.

The method can use various techniques to determine which userinstantiated a tunnel. If the tunnel is a projection of a remote orlocal mobile device, then the tying of authentication can be doneautomatically by passing through the authenticated user on the mobiledevice as the “owner” of the tunnel. If the instantiation of the tunnelis done at the screen on the shared canvas, then the technique can use acombination of mechanisms to determine the identity of the correspondinguser. For example, user tracking mechanisms, such as Kinect, can useoptical and depth sensing cameras. The technique can determine who istouching the screen and where. Facial recognition can identify the user,and through touch identification in the screen touch sensor or sensingpositioning of users correlated with touch points identified by thetouch sensor, the technique can determine who is touching the screen andexactly where and with how many fingers.

The user identity can be tied to the tunnel. Similarly, the presenttechniques can tie the identity of the tunnel creator to any new contentobject created on the shared canvas from the tunnel. For instance, assomeone is drawing free form ink on the shared canvas, the same facialrecognition and touch identification mechanisms can be used to tie themetadata of who created the object to that object. This can enablescenarios like searching for the content by a specific user or by aspecific user in a specific location.

Some implementations can offer a separate affordance on the sharedcanvas in the form of a search box. This search box can be instantiatedeither through a touch affordance or by simply drawing a question markon the shared canvas with ink. Users can use ink, voice, or keyboardentry, among others, to enter their query into this search box. Thesearch box can be used to instantiate content objects (e.g., by asking aquestion that is understood on the device or on backend devices, such asthrough natural language processing (NLP) or other techniques). Thistechnique can instantiate a live dynamic chart to search the web(bringing up a tunnel with a browser showing the web search results), orto search for other shared canvas content that the user has access to(e.g., a shared canvas they worked on last week).

In some configurations, each shared canvas can be fundamentallyaddressed through a uniform resource locator (URL). Shared canvases canbe maintained in a cloud service for retrieval, search, analytics, etc.Metadata for a shared canvas can include the device name on which agroup experience was instantiated as well as the names that weresubsequently opened and edited during the group experience session. Thiscan allow users to search the shared canvas corpus by device, user,content, and/or date/time, among others.

In some implementations, to easily retrieve previous shared canvases,the canvas can have a mechanism to open a gallery that can show previouscanvases for this device (created on this device) and the set ofcurrently recognized users (created on any device, but the current setof users have access). Some configurations can automatically determinethat a previous group of users has returned and present a “welcome backexperience” where the system proactively recognizes (through voice andvision) returning user(s) and automatically sets the device state to thelast session.

The method can detect when individual identified users are no longerinteracting with the graphical user interface. Responsively, the methodcan cease access to the personal data of the individual identified userswho are no longer interacting with the graphical user interface at block1912.

The described methods can be performed by the systems and/or devicesdescribed above relative to FIGS. 17-18, and/or by other devices and/orsystems. The order in which the methods are described is not intended tobe construed as a limitation, and any number of the described acts canbe combined in any order to implement the method, or an alternatemethod. Furthermore, the method can be implemented in any suitablehardware, software, firmware, or combination thereof, such that a devicecan implement the method. In one case, the method is stored oncomputer-readable storage media as a set of instructions such thatexecution by a computing device causes the computing device to performthe method.

To summarize, the present techniques can offer a low friction orfrictionless ability to engage in a group experience on a shared deviceGUI. Any set of users can approach and simultaneously interact with thedevice using intuitive and/or widely recognized user commands.

CONCLUSION

In summary, the shared canvas can allow multiple users to interactsimultaneously. The shared canvas can attempt to recognize users tooffer additional functionality, such as personalized functionalityand/or access to private content. Recognized users can bring theirprivate content into the shared space through a tunnel or othermechanism.

The present concepts can be distinguished from existing sharing whereeach user has their own device to get to shared space, such as during aconference call. Instead, in some implementations of the presentconcepts, the shared canvas does not belong to any of the users. Thegroup experience device supporting the shared canvas can know about theusers that are simultaneously using it either actively and/or viewing aspart of the group. The shared canvas can offer a mechanism forindividual people to get to their private assets and interact with thoseon the shared canvas.

Although techniques, methods, devices, systems, etc., pertaining togroup experience display devices are described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described. Rather,the specific features and acts are disclosed as exemplary forms ofimplementing the claimed methods, devices, systems, etc.

The invention claimed is:
 1. A system, comprising: a display device;sensors configured to detect users that perform user commands proximateto the display device; a processor; and a computer-readable storagemedium storing instructions which, when executed by the processor, causethe processor to: present a shared canvas on the display device; modifythe shared canvas as multiple detected users interact in a collaborativeeffort on the shared canvas via the user commands; determine a relativeconfidence in an identity of an individual user of the shared canvas;detect an individual user command from the individual user; provided therelative confidence in the identity of the individual user issufficient, present a private data portal on the shared canvas for theindividual user in response to the individual user command; display anindividual private data item on the shared canvas responsive to theindividual user identifying the individual private data item via anotherindividual user command directed to the private data portal; and removethe private data portal for the individual user responsive to detectingthat the individual user is no longer interacting with the sharedcanvas.
 2. The system of claim 1, wherein the sensors include a firstset of sensors configured to detect when the users are viewing theshared canvas and a second set of sensors configured to detect the usercommands.
 3. The system of claim 2, wherein either the first set ofsensors or the second set of sensors are configured to detect that theindividual user is looking at a specific location on the shared canvas.4. The system of claim 2, wherein the first set of sensors comprises anarray of cameras and an infrared pattern projector.
 5. The system ofclaim 4, wherein the first set of sensors comprises communicationhardware configured to detect a personal device carried by theindividual user and to utilize information from the personal device toboth detect and identify the individual user.
 6. The system of claim 1,wherein the collaborative effort comprises the multiple detected userssimultaneously interacting with an application via a common area of theshared canvas.
 7. The system of claim 1, wherein the instructions, whenexecuted by the processor, cause the processor to: obtain a copy of theindividual private data item from a personal device of the individualuser, the copy being displayed on the shared canvas.
 8. The system ofclaim 7, wherein the instructions, when executed by the processor, causethe processor to: retain the copy of the individual private data itemobtained from the personal device of the individual user on the sharedcanvas after the private data portal has been removed from the sharedcanvas.
 9. The system of claim 1, wherein the instructions, whenexecuted by the processor, cause the processor to: obtain a copy of theindividual private data item from a cloud device, the copy beingdisplayed on the shared canvas.
 10. The system of claim 9, wherein theinstructions, when executed by the processor, cause the processor to:retain the copy of the individual private data item obtained from thecloud device on the shared canvas after the private data portal has beenremoved from the shared canvas.
 11. A system, comprising: a display; aprocessor; and, a computer-readable storage medium storingcomputer-readable instructions which, when executed by the processor,cause the processor to: detect multiple users; present a graphical userinterface on the display; allow the multiple users to simultaneouslyinteract with the graphical user interface in a collaborative effortutilizing basic functionality; attempt to identify the multiple users bydetermining respective confidence levels in identities of the multipleusers; designate certain users having associated confidence levels thatexceed a threshold as identified users; enable advanced functionality tothe identified users, wherein the advanced functionality includesdisplaying, on the graphical user interface, a user interface elementthat provides access to personal data of an individual identified user;detect when the individual identified user is no longer interacting withthe graphical user interface; and responsive to detecting that theindividual identified user is no longer interacting with the graphicaluser interface, remove the user interface element that provides accessto the personal data of the individual identified user from thegraphical user interface.
 12. The system of claim 11, further comprisingmultiple sensors and wherein the computer-readable instructions furthercause the processor to cause the multiple sensors to detect the multipleusers.
 13. The system of claim 11, wherein the basic functionalitycomprises modifying a shared canvas shown on the display responsive todrawing or text inputs provided by the multiple users.
 14. The system ofclaim 11, wherein the computer-readable instructions further cause theprocessor to: only enable the advanced functionality for specificidentified users having corresponding assigned confidence scores thatsatisfy a confidence threshold obtain biometric features of the multipleusers from one or more sensors; and use the biometric features todetermine the respective confidence levels in the identities of themultiple users.
 15. The system of claim 14, wherein thecomputer-readable instructions further cause the processor to: providean individual aspect of the advanced functionality to the certain usershaving the associated confidence levels that exceed the confidencethreshold, and provide another individual aspect of the advancedfunctionality to a subset of the certain users that have correspondingconfidence levels exceeding another confidence-threshold that is higherthan the confidence threshold.
 16. The system of claim 11, wherein thecomputer-readable instructions, when executed by the processor, causethe processor to: determine that the individual identified user is nolonger interacting with the graphical user interface by detecting thatthe individual identified user has walked away from the display, theuser interface element being removed responsive to detecting theindividual identified user walking away from the display.
 17. A method,comprising: sensing, via sensors, user commands of a collaborativeeffort performed by multiple users relative to a graphical userinterface of a display device; determining an identification of anindividual user that performed an individual sensed user command of thecollaborative effort; obtaining a confidence level of the identificationof the individual user; making a determination whether to implement theindividual sensed user command on the graphical user interface based atleast in part on the confidence level; in at least one instance,implementing the individual sensed user command based at least in parton the determination by displaying a user interface element thatprovides the individual user with access to private data stored remotelyfrom the display device; obtaining a copy of at least one private dataitem of the individual user and displaying the copy on the displaydevice; and responsive to detecting that the individual user is nolonger interacting with the graphical user interface, removing the userinterface element from the graphical user interface.
 18. The method ofclaim 17, wherein the sensors comprise biometric sensors.
 19. The methodof claim 17, wherein the determining the identification of theindividual user is performed solely from information obtained from theindividual sensed user command.
 20. The method of claim 17, furthercomprising detecting a personal device of the individual user anddetermining the identification of the individual user based at least inpart on information obtained from the personal device.
 21. The method ofclaim 17, further comprising determining whether to implement theindividual sensed user command by comparing the confidence level to aconfidence threshold defined for an application program interface thatis configured to implement the individual sensed user command.
 22. Themethod of claim 17, implemented by the display device, wherein thedisplay device is manifest as a group experience display device thatincludes the sensors.
 23. The method of claim 17, wherein the usercommands of the collaborative effort comprise interactions of themultiple users with a common area of a shared canvas.